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1 CLAIMS 

2 

3 What is claimed is: 

4 

5 1 . A computer-implemented method of reducing graphical user 

6 interface (GUI) noise comprising: 

7 recording a first execution scenario for control of operation of an 

8 application program having a GUI during a recording phase of operation of a 

9 cognitive control framework system; 

1 0 setting soft conditions for a search for the application program for the first 

1 1 execution scenario; 

12 playing back the application program according to the first execution 

13 scenario during a playback phase of operation of the cognitive control framework 

14 system; 

1 5 updating the first execution scenario to form a second execution scenario 

16 to reduce GUI noise conditions observed during playback, including updating 

17 recorded images originally generated by the GUI during the recording phase and 

1 8 updating coordinates for user input data; 

19 setting stronger conditions for the search for use in subsequent 

20 playbacks; and 

21 playing back the application program according to the second execution 

22 scenario with the stronger conditions for search. 
23 

24 2. The method of claim 1 , wherein the soft conditions comprise a first set 

25 of bounds for differences in shapes of contours, text and image content, or 

26 layout. 

27 

28 3. The method of claim 2, where the stronger conditions comprise a 

29 second set of bounds for differences in shapes of contours, text and image 

30 content, or layout, the second set being different than the first set. 
31 
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32 4. The method of claim 1 , wherein the user input data comprises mouse 

33 selections. 
34 

35 5. The method of claim 1 , wherein GUI noise conditions comprise at least 

36 one of changeable color schemes, highlighting of items, noise from video 

37 sources, and anti-aliasing effects. 
38 

39 6. The method of claim 1 , wherein updating recorded images comprises 

40 using playback images as recorded images for subsequent playbacks. 
41 

42 7. An article comprising: a machine accessible medium containing 

43 instructions, which when executed, result in reducing graphical user interface 

44 (GUI) noise by 

45 recording a first execution scenario for control of operation of an 

46 application program having a GUI during a recording phase of operation of a 

47 cognitive control framework system; 

48 setting soft conditions for a search for the application program for the first 

49 execution scenario; 

50 playing back the application program according to the first execution 

51 scenario during a playback phase of operation of the cognitive control framework 

52 system; 

53 updating the first execution scenario to form a second execution scenario 

54 to reduce GUI noise conditions observed during playback, including updating 

55 recorded images originally generated by the GUI during the recording phase and 

56 updating coordinates for user input data; 

57 setting stronger conditions for the search for use in subsequent 

58 playbacks; and 

59 playing back the application program according to the second execution 

60 scenario with the stronger conditions for search. 
61 
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62 8. The article of claim 7, wherein the soft conditions comprise a first set of 

63 bounds for differences in shapes of contours, text and image content or layout. 
64 

65 9. The article of claim 8, where the stronger conditions comprise second 

66 set of bounds for differences in shapes of contours, text and image content, or 

67 layout, the second set being different than the first set. 
68 

69 10. The article of claim 7, wherein the user input data comprises mouse 

70 selections. 
71 

72 11. The article of claim 7, wherein GUI noise conditions comprise at least 

73 one of changeable color schemes, highlighting of items, noise from video 

74 sources, and anti-aliasing effects. 
75 

76 12. The article of claim 7, wherein instructions to update recorded images 

77 comprise instructions to use playback images as recorded images for 

78 subsequent playbacks. 
79 

80 13. A method of automatically controlling execution of an application 

81 program having a GUI to reduce GUI noise comprising: 

82 capturing user input data and images displayed by the GUI during a 

83 recording phase of execution of the application program; 

84 analyzing the captured user input data and recorded images to generate a 

85 first execution scenario during the recording phase; 

86 setting soft conditions for a search for the application program for the first 

87 execution scenario; 

88 generating simulated user input data based on the first execution scenario 

89 during a playback phase of execution of the application program and inputting 

90 the simulated user input data to the application program; 
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91 performing image analysis on playback images displayed by the GUI as a 

92 result of processing the simulated user input data during the playback phase and 

93 the recorded images; 

94 updating the first execution scenario to form a second execution scenario 

95 to reduce GUI noise conditions observed during playback, including updating the 

96 recorded images originally generated by the GUI during the recording phase and 

97 updating coordinates for user input data; 

98 setting stronger conditions for the search for use in subsequent 

99 playbacks; and 

100 playing back the application program according to the second execution 

1 01 scenario with the stronger conditions for search. 
102 

103 14. The method of claim 13, wherein the soft conditions comprise a first 

1 04 set of bounds for differences in shapes of contours, text and image content, or 

105 layout. 
106 

107 15. The method of claim 14, where the stronger conditions comprise a 

108 second set of bounds for differences in shapes of contours, text and image 

109 content, or layout, the second set being different than the first set. 
110 

111 16. The method of claim 14, wherein GUI noise conditions comprise at 

112 least one of changeable color schemes, highlighting of items, noise from video 

113 sources, and anti-aliasing effects. 
114 

115 17. The method of claim 14, wherein updating recorded images 

116 comprises using playback images as recorded images for subsequent 

117 playbacks. 
118 

119 18. An article comprising: a machine accessible medium containing 

120 instructions, which when executed, result in automatically controlling execution of 

121 an application program having a GUI to reduce GUI noise by 



29 



WO 2007/055612 



PCT/RU2005/000559 



122 capturing user input data and images displayed by the GUI during a 

123 recording phase of execution of the application program; 

124 analyzing the captured user input data and recorded images to generate a 

125 first execution scenario during the recording phase; 

126 setting soft conditions for a search for the application program for the first 

127 execution scenario; 

128 generating simulated user input data based on the first execution scenario 

129 during a playback phase of execution of the application program and inputting 

130 the simulated user input data to the application program; 

131 performing image analysis on playback images displayed by the GUI as a 

132 result of processing the simulated user input data during the playback phase and 

1 33 the recorded images; 

134 updating the first execution scenario to form a second execution scenario 

135 to reduce GUI noise conditions observed during playback, including updating the 

136 recorded images originally generated by the GUI during the recording phase and 

1 37 updating coordinates for user input data; 

138 setting stronger conditions for the search for use in subsequent 

139 playbacks; and 

140 playing back the application program according to the second execution 

141 scenario with the stronger conditions for search. 
142 

143 19. The article of claim 18, wherein the soft conditions comprise a first set 

144 of bounds for differences in shapes of contours, text and image content, or 

145 layout. 
146 

147 20. The article of claim 19, where the stronger conditions comprise a 

148 second set of bounds for differences in shapes of contours, text and image 

149 content, or layout, the second set being different than the first set. 
150 
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151 21. The article of claim 18, wherein GUI noise conditions comprise at 

152 least one of changeable color schemes, highlighting of items, noise from video 

153 sources, and anti-aliasing effects. 
154 

155 22. The article of claim 18, wherein instructions to update recorded 

1 56 images comprise instructions to use playback images as recorded images for 

1 57 subsequent playbacks. 
158 
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